Three supervised learning methods on pen digits character recognition dataset

Supervised learning is a broad field that encompasses a number of methods, which can generally be classificed into two categories: parametric and nonparametric. In the parametric methods, it is assumed that the forms of the underlying density functions are known. The problem of estimating unknown functions can be reduced to estimating some values of parameters. In contrast, in the nonparametric method, there is no assumption on the form of the underlying densities. The parametric category is divided further into two subcategories: generative and discriminative. In the generative method, we estimate P(X|Y), which describes how to generate X given Y, while in the discriminative method, we directly estimate P(Y|X). It is our goal to compare classification results and characteristics of learning methods in the different categories. Bayesian classification with a mixture of Gaussians, logistic regression, and k nearest-neighbor classification were implemented, and their results using a pen digits character recognition dataset are analyzed.